140 research outputs found
Equality in the Matrix Entropy-Power Inequality and Blind Separation of Real and Complex sources
The matrix version of the entropy-power inequality for real or complex
coefficients and variables is proved using a transportation argument that
easily settles the equality case. An application to blind source extraction is
given.Comment: 5 pages, in Proc. 2019 IEEE International Symposium on Information
Theory (ISIT 2019), Paris, France, July 7-12, 201
Stochastic Interpretation for the Arimoto Algorithm
The Arimoto algorithm computes the Gallager function for a given channel and parameter
, by means of alternating maximization. Along the way, it generates a
sequence of input distributions , , ... , that
converges to the maximizing input . We propose a stochastic
interpretation for the Arimoto algorithm. We show that for a random (i.i.d.)
codebook with a distribution , the next distribution
in the Arimoto algorithm is equal to the type () of the
feasible transmitted codeword that maximizes the conditional Gallager exponent
(conditioned on a specific transmitted codeword type ). This
interpretation is a first step toward finding a stochastic mechanism for
on-line channel input adaptation.Comment: 5 pages, 1 figure, accepted for 2015 IEEE Information Theory
Workshop, Jerusalem, Israe
Multiple-Description Coding by Dithered Delta-Sigma Quantization
We address the connection between the multiple-description (MD) problem and
Delta-Sigma quantization. The inherent redundancy due to oversampling in
Delta-Sigma quantization, and the simple linear-additive noise model resulting
from dithered lattice quantization, allow us to construct a symmetric and
time-invariant MD coding scheme. We show that the use of a noise shaping filter
makes it possible to trade off central distortion for side distortion.
Asymptotically as the dimension of the lattice vector quantizer and order of
the noise shaping filter approach infinity, the entropy rate of the dithered
Delta-Sigma quantization scheme approaches the symmetric two-channel MD
rate-distortion function for a memoryless Gaussian source and MSE fidelity
criterion, at any side-to-central distortion ratio and any resolution. In the
optimal scheme, the infinite-order noise shaping filter must be minimum phase
and have a piece-wise flat power spectrum with a single jump discontinuity. An
important advantage of the proposed design is that it is symmetric in rate and
distortion by construction, so the coding rates of the descriptions are
identical and there is therefore no need for source splitting.Comment: Revised, restructured, significantly shortened and minor typos has
been fixed. Accepted for publication in the IEEE Transactions on Information
Theor
Joint Wyner-Ziv/Dirty Paper coding by modulo-lattice modulation
The combination of source coding with decoder side-information (Wyner-Ziv
problem) and channel coding with encoder side-information (Gel'fand-Pinsker
problem) can be optimally solved using the separation principle. In this work
we show an alternative scheme for the quadratic-Gaussian case, which merges
source and channel coding. This scheme achieves the optimal performance by a
applying modulo-lattice modulation to the analog source. Thus it saves the
complexity of quantization and channel decoding, and remains with the task of
"shaping" only. Furthermore, for high signal-to-noise ratio (SNR), the scheme
approaches the optimal performance using an SNR-independent encoder, thus it is
robust to unknown SNR at the encoder.Comment: Submitted to IEEE Transactions on Information Theory. Presented in
part in ISIT-2006, Seattle. New version after revie
Random Subsets of Structured Deterministic Frames have MANOVA Spectra
We draw a random subset of rows from a frame with rows (vectors) and
columns (dimensions), where and are proportional to . For a
variety of important deterministic equiangular tight frames (ETFs) and tight
non-ETF frames, we consider the distribution of singular values of the
-subset matrix. We observe that for large they can be precisely
described by a known probability distribution -- Wachter's MANOVA spectral
distribution, a phenomenon that was previously known only for two types of
random frames. In terms of convergence to this limit, the -subset matrix
from all these frames is shown to be empirically indistinguishable from the
classical MANOVA (Jacobi) random matrix ensemble. Thus empirically the MANOVA
ensemble offers a universal description of the spectra of randomly selected
-subframes, even those taken from deterministic frames. The same
universality phenomena is shown to hold for notable random frames as well. This
description enables exact calculations of properties of solutions for systems
of linear equations based on a random choice of frame vectors out of
possible vectors, and has a variety of implications for erasure coding,
compressed sensing, and sparse recovery. When the aspect ratio is small,
the MANOVA spectrum tends to the well known Marcenko-Pastur distribution of the
singular values of a Gaussian matrix, in agreement with previous work on highly
redundant frames. Our results are empirical, but they are exhaustive, precise
and fully reproducible
Anti-Structure Problems
The recent success of structured solutions for a class of
information-theoretic network problems, calls for exploring their limits. We
show that sum-product channels resist a solution by structured (as well as
random) codes. We conclude that the structured approach fails whenever the
channel operations do not commute (or for general functional channels, when the
channel function is non decomposable).Comment: a short note, following the Banff meeting on Algebraic structure in
network information theroy, Aug. 14-1
- …